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INTRODUCTION 

Early in the history of computers, programmers found 
that their programs were increasing in size until they were 
literally larger than the addressing range of the computer. To 
solve this problem, a technique called virtual memory was in- 
vented. This technique allows the programmer to use a larger 
address space for his programs than the physical address 
space of the main memory by automatically storing and 
retrieving parts of the program in secondary memory (usually 
a disk). 

The original virtual memory technique, as implemented by 
IBM, used a main memory addressing scheme which 
referenced a page table to get a pointer into a block table. In 
turn, this pointer was used to form the physical address. This 
meant that every memory reference required three main 
memory references. The use of a cache of addresses in an 
associative memory can cut this time significantly. 

This paper presents a design for a virtual memory machine 
using the currently available MC68000 microprocessor 
(MPU) and the MC68451 memory management unit 
(MMU). The presentation includes a discussion on some 
problems inherent to virtual memory design and the methods 
used to resolve these problems. 

DESIGN GOALS 

In this design, the user program (called a task) is allowed 
to request and receive from the operating system more 
memory than is physically available. The operating system 
then allocates some minimum amount of memory to the task 
and constructs a segment or segments in the MMU to 
describe the page allocated. If the task then tries to access 
memory which has not been physically allocated, an undefin- 
ed segment acces s error is generated by the MMU. The MMU 
then asserts the FAULT signal to indicate that a page fault 
has occurred. 

Once a page fault has occurred, some mechanism must be 
available to locate and fix the fault. This consists of deter- 
mining if the present page has been modified and, if it has, to 
save it on the disk. The new page containing the location 
whose address caused the page fault must then be loaded 



from the disk into memory. The segment descriptor(s) which 
describe the page in the MMU must then be modified to 
reflect the new memory configuration. 

Two approaches toward the achievement of these goals are 
presented. These are the bus cycle rerun method and the bus 
cycle suspension method. 



BUS CYCLE RERUN METHOD 

The obvious candidate to fix the page fault is the MPU, as 
it has access to both the MMU and the DMAC. Unfortunate- 
ly, the bus cycle which caused the fault must be rerun after 
the fault has been fixed in order to continue executing the 
progra m. Th e MC68000 can rer un bus cycles by using the bus 
error (BERR) and halt (HALT) signals. However, the MPU 
is in the halt state between the aborted cycle and the rerun cy- 
cle and cannot fix a page fault while it is halted. Therefore, 
another bus master must perform this function and, since 
this bus master and the main MPU can share memory 
management routines, this could be an MC68000 MPU as 
well. 

A block diagram of a bus rerun type of system is shown in 
Figure 1 . The MPU labeled Executor serves as the main pro- 
cessor, executing the operating system and the user tasks. 
The Fixer is responsible for fixing page faults. Since both 
MPUs share a common bus, the bus request (BR) and bus 
grant (BG) control signals are used by the control logic to 
allow only one MPU at a time to u se the b us. 

When a page fault occurs, the BERR, HALT, and BR 
lines on the Executor are asserted. This causes the current 
bus cycle to be terminated and the Executor to be halted. As 
soon as the cycle terminates, the MPU relinquishes the bus. 
The control logic then releases the Fixer by negating its BR 
line and the Fixer takes control of the bus and fixes the page 
fault. After resolving the page fault, the Fixer writes to a 
special location to toggle a flip-flop which causes the swap to 
occur. The BR line on the Fixerjs^ asserted and it is removed 
from the bus. The HALT and BR lines on the Executor are 
then negated and the Executor performs the bus rerun and 
then continues executing the user task. 
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While this method is relatively conservative of hardware, it 
does have one major drawback. In order to preserve the inte- 
grity of semaphores useful in multi-processor applications, 
read-modify-write bus cycles cannot be rerun. In practice, 



this means that user applications programs may not use the 
test and set (TAS) instructions in the bus cycle rerun method. 
Since it may not be possible to apply this restriction, par- 
ticularly on vendor-supplied software, another method is 
proposed. 
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FIGURE 1 — Bus Cycle Rerun System Block Diagram 



BUS CYCLE SUSPENSION METHOD 

The MC68000 has an asynchronous bus interface. The 
MPU asserts address strobe (AS) to indicate a valid address 
on the add ress b us and one of the upper or lower data strobes 
(UDS) or (LDS) to indicate valid data during a write cycle. 
The MP U then expects a data transfer acknowledge 
(DTACK) response signal to be asserted, indicating that the 
data has been accepted (write) or is valid (read). If DTACK 
has not been asserted by the falling edge of state four (S4) of 
the syste m clock, the MPU idles, inserting wait states until 
DTA CK is as serted. The bus cycle is therefore suspended 
until DTACK is asserted and this delay can be used by the 
Fixer to fix the page fault. 

A block diagram of a system using this method is shown in 
Figure 2. Since the Executor is driving the address, control 
and, possibly, the data buses during the "suspension," three- 
state buffers are needed to isolate these signals from the 
system bus while the Fixer is active. The Fixer is held off the 
bus while the Executor is active with the BR signal. This 
signal causes all buses and control signals on the Fixer to 
enter the high-impedance state and to halt. 

When the Executor executes a bus cycle wherein a page 
fault occurs, the MMU withholds the mapped address strobe 



(MAS) and asserts the FAULT signal. This action disables 
the three-state buffers and removes the Executor from the 
system bus . Since t he data transfer acknowledge line on the 
Executor (DTACK(E)) is held negated by a p ullup resi stor, 
the Executor idles in the wait state. Asserted FAULT also 
negates the BR line to the Fixer and releases the Fixer to con- 
trol the bus. After performing the fix, the Fixer writes to a 
selected location to cause a swap. Signal BR(F) is again 
asserted, removing the Fixer from the bus and allowing the 
suspende d bus access of the Executor to be completed. Signal 
DTACK(E) is then asserted by the addressed memory block 
or peripheral and the cycle wherein the page fault occurred 
terminates. The Executor then continues with the user task. 

The tradeoff in this method is the amount of hardware re- 
quired versus a versatile instruction handling capability. 
Although the control logic is less complex, three-state buffers 
are required for the address, data, and control buses of the 
Executor. Also, multiplexers are needed for the address and 
bus request lines. However, all instructions, including TAS, 
can be used on this system. Thus, this method is preferred as 
it results in a more powerful and versatile system. A design 
incorporating this method is described in detail in the follow- 
ing paragraphs. 
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FIGURE 2 — Suspended Bus Cycle System Block Diagram 



A DESIGN USING THE CYCLE 
SUSPENSION METHOD 

Figure 3 shows a schematic diagram of a virtual memory 
machine using two MC680OOL8 microprocessors. The Ex- 
ecutor is isolated from the system bus_by three-state buffers. 
These buffers are controlled by the E/F signal generated by 
control logic flip-flops Ul and U2. When E/F is low, the 
buffersjire enabled and the Executor is in control of the bus. 
When E/F is high, the buffers are in the high-impendance 
state and the Executor is removed from the bus. The control 
logic uses the SWAP-BR signal to remove the Fixer from the 
bus while the Executor is processing. 

The system address strobe, AS(S), is derived from the 
multiplexed address strobes of the Executor (AS(E)) and the 
Fixer (A S(F)) . The address control logic flip-flops U3 and U4 
use the ASE/ASF signal to select either AS(E ) or AS(F) as 
the system address strobe AS(S). When ASE/ASF is low, 
AS(S) is AS(E) and when it is high, AS(S) is AS(F). This 
signal is asserted one clock cycle after E/F to allow for 
address setup time to the MMU. 



The bus request handshake lines, BR, BG, and BGACK 
are similarly multiplexed with gates U18 through U26 to 
allow the DMAC to request the bus from the current_pro- 
cessor in control. The sel ect line of this multiplexer is E/F. 

The MMU can assert FAULT for an undefined segment 
access (USA) and a write violation (WV). This creates a 
problem in that in this system, a USA may or may not be an 
error. If the task is attempting to access memory granted to it 
by the operating system but not physically allocated (virtual 
memory) then this constitutes a page fault and the Fixer can 
resolve it. If the access is to a location not requested by the 
task, the USA is an error and an exception must be forced on 
the Executor. Similarly, a write violation is an attempt by a 
user task to write a write-protected segment. This is also an 
error and must be terminated. The write violation error 
(WVERR) signal is used to resolve this problem. 

The Fixer can assert the WVERR signal by performing a 
read at a spec ified location. In turn, WVERR asserts the Ex- 
ecutor BERR line, forcing the Executor to abort the access 
and take the bus error exception when it regains the bus. 
Thus, the Fixer is allowed to deal with true USA and WV 
errors. 
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An MC68230 Parallel Interface/Timer (PUT) is provided 
as a watchdog timer to terminate any accesses to unpopu- 
lated addresses. In addition, the PI/T has a numbe r of 
"null" registers which, when accessed, return DTACK with 
no other effect. The "null" register at location $1F is used by 
the control logic to allow the Fix er to cause an MPU switch. 
A write to this location causes SWAP-BR to be asserted, 
causing the Fixer to relinquish the bus. A read of the same 
location asserts BERR to the Executor. If the PI/T is not in- 
cluded, some means of generating DTACK for the switch 
register must be used. 

OPERATIONAL DESCRIPTION 

Resetting the System 

The system must be initialized before it can begin opera- 
tion. This is accomplished by resetting both processors. 
Refer to the schematic in Figure 3 and the timing diagram in 
Figure 4 for the foll owing. 

First the RESET and HALT lines of both MPUs are 
asserted by the external reset signal. This sets flip-flops Ul 
and U2 via ANDgate U5. The outputs of these _flip- flops are 
SWAP-BR and E/F, respectively. A hig h on E/F, in turn, 
sets flip-flops U3 and U4 forcing ASE/ASF high. In this 
state, the buffers from the Executor are disabled and th e 
Fixer is in control of the system bus. When RESET/HALT is 
negated, the Fixer fetches its restart vectors and begins execu- 
tion of the boot ROM. 

Th e fixer m ust first initialize the MMU. Since E/F is ORed 
with BGACK from the DMAC, it shares the second set of 
eight entries in the address space table. Hence, the Fixer 
should load a descriptor which corresponds to the operating 
system segment for the Executor. Then, when control is 
switched to the Fixer during a fault, its address space allows 
it to address and execute the fixup routines. 

After setting up the MMU, the Fixer then writes to the 
switch location in the PI/ T. A write to $1F causes the output 
of U7 to go low, asse rting SWITCH. This clears flip-flo p U2, 
asse rting SW AP-BR. When this write cycle is finished, MAS 
and DTACK(S) are negated and clock flip-flop Ul. The out- 
put of Ul, E/F, goes low, enabling the buffers from the Ex- 
ecutor. When E/F goes low, flip-flops U3 and U4 are releas- 
ed from preset and ASE/ASF goes low after two rising edges 
of the system clock. This allows an address setup time before 
AS(S) is asserted . 

After RESET/HALT is negated, the Executor begins a 
read bus cycle to fetch its restart vector and waits for 
DTACK(E) to be asserted. However, its buffers are disabled 
while the Fixer initializes the MMU. Then, after its buffers 
are enabled, a valid address is put on the system address bus 
and AS(S) i s asserted . The read is completed and the memory 
unit asserts DTACK(E). The Executor then begins execution. 
First, it initializes the MMU and then loads the operating 
system. User tasks are then processed using the loader and 
other system utilities. 

Page Fault 

Refer to the timing diagram in Figure 5 and assume that 
the Executor is running in user mode and is executing a user 
task. It attempts to read from a logical address not presently 
in physical memory. The MMU de tects an un define d seg- 
ment access and asserts the FAULT line. S ignal MA S is not 
asserted , so memory does not return DTA CK. When 
FAULT goesjow, Ul and U2 are preset, negating SWAP-BR 
and forcing E/ F high. T his removes the executor from the 
bus and blocks DTACK. The Fixer, with BR released, starts 
a bus cycle in approximately 3 clock cycles. 



The Fixer then reads the MMU to determine what sort of 
fault caused the switch. If it was a bona fide page fault, it 
checks the M (modified) bit in the segment status register to 
see if the segment had been written to. If so, the DMAC is 
programmed to write the page to the disk. The page with the 
address of the suspended bus cycle is then loaded into 
memory from the disk. Then the page table is updated to 
reflect the new logical base address of the segment(s). That 
done, the page fault fix cycle is complete. 

To accomplish the processor switch, the Fixer then writes 
to the switch register and the switch sequence described 
above is initiated. The buffers are enabled and the address 
strobe multiplexer is switched to allow the asserted AS(E) to 
reach the MMU. The MMU does the translation, the 
memory unit performs the access and asserts DTACK, and 
the suspended cycle is completed. 

Undefined Segment Accesses and Write Violations 

The last case to be covered is that of a genuine error on the 
part of the user program. If the user task attempts a memory 
reference outside of its defined address space or attempts to 
write to a location that has been defined as read- only by the 
operatin g system, the MMU does not assert MAS but asserts 
FAULT. Refer to the timing diagram in Figure 6 and assume 
that the Executor is in control of the bus and is attempting to 
write to a write-protected segment. The MMU asserts 
FAULT causing the switch to the Fixer. The Fixer then reads 
the segment status register and finds that the fault was caused 
by a WV and the Fixer then executes a test-and-set (TAS) in- 
struction on the s witch regis ter. The read portion o f the TAS 
instruction ca uses WVER R to be asserted. In turn WVERR 
generates th e WV-BER R signal at th e Q ou tput of flip-flop 
U27. Signal WV-BERR then assserts BERR on the Executor 
via gate U15. The Executor then terminates the suspended 
write cycle . When AS(E) is negated, U27 is preset and the 
WV-BERR signal is negated. The Executor then begins 
stacking f or the bu s error exception but, since the buffers are 
disabled, DTACK(E) is not returned and the MPU waits. 

The write porti on of the T AS instruction executed by the 
Fixer then asser ts SW ITCH and the Fixer is removed from 
the bus. When ASE/ASF goes high, AS(S) is asserted and 
the write for the stack operation of the Executor is made. 
Signal DTACK(E) is asserted and the bus error handling 
routine is taken. 

SOFTWARE 

The routines to drive this system are not uncommon to 
operating systems. Many of the same memory management 
routines necessary to a multi-tasking system can be shared 
between the Fixer and the Executor. The flow chart for an 
example driver routine is given in Figure 7. Although the 
Fixer could be used to execute any of the operating system 
functions, no advantage would be gained. However, if a 
slower, less expensive MPU is used, overhead is minimized if 
the code which it has to execute is also minimized. 

CAUTIONS 

This implementation assumes that the entire machine, in- 
cluding the software, is a finite machine. In particular, it has 
been assumed that the DMAC is not made active if the 
possibility of a page fault occurring exis ts. If the BR line 
from the DMAC is asynchronous with the FAULT line from 
the MMU, there exists the possibility of the system hanging. 
This depends on the implementation of the DMAC bus arbi- 
tration circuits, as it is possible to supply a BG signal without 
a BR from the DMAC. 
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FIGURE 7 — System Functional Flow Chart 
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